The operations of many modern processors are pipelined so as to improve the speed of the processor. Thus, a second load operation will enter a load/store pipeline before a first operation will pass completely through the pipeline. Typically, a cache memory which loads data to a register or stores data from the register is outside of the load/store pipeline. When an operation is passing through the load/store pipeline, the cache memory will be accessed. If the proper data is in the cache memory at the correct address, a hit is returned, and the data is loaded to the registers and the pipelining of operation continues. When requested data is not in the cache memory, a miss is returned and the data must be fetched from the main memory over an external bus. Until the miss is serviced such that the data is retrieved from main memory and loaded to the cache, the pipeline is stalled. Since a retrieval from main memory can be relatively slow compared to the pipeline operation, if there are many cache misses, the load/store operation will be dominated by the servicing of those misses. This loses the performance advantages sought by pipelining.
Vector processors are designed to operate on arrays of information in an extremely fast manner. These vector processors typically include a vector register file that is a data store, which is coupled to arithmetic units which operate on the data stored in the vector register file. Data is loaded into the vector register file from a memory, such as a cache memory. The cache memory can also receive and store data from the vector register file. The cache memory itself receives data from a large, main memory, which is typically coupled by a bus to the vector processor. When data is to be loaded into the vector register file, a load/store pipeline accesses the cache memory for the data elements and sends them over a bus to the vector register file.
Scalar processors in which instructions are pipelined would also benefit from the use of a cache which is integral to the pipeline. Again, the problem of multiple cache misses can cause the pipeline to be dominated by the servicing of those misses.